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ABSTRACT 



Apparatus is described for processing a digitized image 
signal obtained firom an image sensor having color photo- 
sites aligned in rows and columns that generate at least three 
separate color values but only one color value for each 
photositc location, structure for interpolating color values 
for each photosite location so that it has three different color 
values. The apparatus includes a memory for storing the 
digitized image signal and a processor operative with the 
memory for generating an appropriate color value missing 
from a photosite location by the interpolation of an addi- 
tional color value for such photosite locations from color 
values of different colors than the missing color value at 
nearby photosite locations. The processor also includes 
structure for obtaining Lapladan second-order values and 
gradient values in at least two image directions &om nearby 
photosites of the same colunm and row and for adding the 
Lapladan second-order values and the gradient values to 
define a classifier and for selecting a preferred orientation 
for the interpolation of the missing color value based upon 
a dassifiex. Finally, an arrangement is provided for interpo- 
lating the missing color value firom nearby multiple color 
values selected to agree witii the preferred orientation. 

14 Claims, 5 Drawing Sheets 
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AND VERTICAL COMPOSITE 
CLASSIFIER VALUES 



/ 



50 



52 



ARE THE aASSIFIER 
VALUES EQUAL? 



NO 
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SELECT THE INTERPOLATION 
METHOD CORRESPONDING 
TO THE SMALLER VALUES 



YES ^ 
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ADAPTIVE COLOR PLAN INTERPOLATION nieans for obtaining Lapladan second-order values and 

IN SINGLE SENSOR COLOR ELECTRONIC gradient values in at least two image directions from 

CAMERA nearby photosites of the same column and row; 

„ _ „ «™ means for adding the Laplacian second-order values and 

CROSS REFERmCE TO RELAIBD 5 the gradient values to define a classifier and for sdect- 

APPLICAnONS ^ preferred orientation for the interpolation of the 

Reference is made to commonly assigned U.S. Ser, No. missing color value based upon a classifier; and 

08/407,436 filed Mar. 17, 1995, issued as U.S. Pat. No. means for interpolating the missing color value firom 

5,506,619 on Apr. 9, 1996 to James E. Adams, Jt. and John nearby multiple color values selected to agree with the 

F. Hamilton, Jr. filed concurrently hwwith, the teachings of preferred orientatioa 



which are incorporated herein. 

HELD OF THE INVENTION 



Advantages 



The advantages of this invention are 1) is computationally 
This invention relates to the field of electronic imaging cfifidcntboth in execution time and memory storage require- 
and is particularly suitable to electronic still imaging by ^^^^45. 2) by use of the combination of the Laplacian 
means of an electronic still camera having a single color second-order values and the gradient values to produce a 
sensor and memory. classiiicr, artifacts (color interpolation) in ou^t image are 

BACKGROUND OF THE INVENHON substantially reduced. 

In electronic color imaging, it is desirable to simulta- ^ BRIEF DESCRBPTION OF THE DRAWINGS 
neously capture image data in Aree color planes, usuaUy no. 1 is a block diagram of an electronic still camera 
red, green and blue. When the three color planes are employing interpolation processing according to flic inven- 
combincd, it is possible to create high-quality color images. ^^j^. 

C^turing these three sets of image data can be done in a *_ ^ . , , . 1 • jr*u : 

o r t ^ • tT ^ t- • 2^ FIG. 2 IS a block diagram of the logic of the interpolation 

number of ways. In electromc photography, this is some- xx^j. « w*vvn. ««^oi« ^ 

..'.J, . • 1 ^r: j-ll processing techmque used m connection with the mvention; 

tunes accomphshed by usmg a smgle two dimensional array f & n . ^ ^ 

of sensors that are covered by a pattern of red, green and FIG* 3 is a detailed block diagram of the logic of the 
blue filters. This type of sensor is known as a color filter interpolation lummancc section m FIG. 2: 
array or CFA. Below is shown the red (R), green (G) and FIG. 4 is a more detailed block diagram of the logic of the 
blue (B) pixels as are commonly arranged on a CFA sensor. chrominance section in FIG. 2; and 

When a color image is captured using a CFA, it is FIGS. 5 and 6 show logic block diagrams for the inter- 
necessary to interpolate the red, green and blue values so polation processes in accordance with the invention, 
that there is an estimate of aU three color values for each 

sensor location. Once the interpolation is done, each picture 35 DETAMD UE^S^m>N OF THE 

dement, or pixel, has tiiree color values and can be pro- INVENTION 
cessed by a variety of known image processing techniques since singlc-scnsor electronic cameras eii^loying color 
depending on the needs of the system. Some exaiiq)les of the filter arrays are weU known, the present description will be 
reasons for processing are to do image sharpening, colcsr directed in particular to elements forming part of, or coop- 
coirection or halftoning. 40 crating more directly wifli, apparatus and method in accor- 

The following shows how red green and blue pixels can dance with the present invention. Elements not specifically 
be arranged in a color filter array. For a more detailed shown or described herein may be selected from those 
description see U.S. Pat No. 3,971,065 to Bayer. known in the art 

Referring initially to FIGS. 1 and 2, an electronic still 
OKQK 45 camera is divided generally into an input section 2 and an 

interpolation and recording section 4. The input section 2 
includes an exposure section 10 for directing image light 
from a subject (not shown) toward an image sensor 12. 
Althouglh not shown, the exposure section 10 includes 
SUMMARY OF INVENTION 50 conventional optics for directing the image light through a 

. . . ^ diaphragm, whldi regulates the optical aperture, and a 

The object of this mvention is to provide an unprov«i shutter, which regulates exposure time. The image sensor 12, 
apparatus for estimating the missing pixel values m a CFA. ^^^^ ^ two^ensional array of photosites cor- 

This object is achieved in apparatus for processing a responding to picture elements of flie image, is a conven- 
digitized image signal obtained firom an image sensor having 55 ^^j^^ charge-coupled device (CCD) using either well- 
color photosites aligned in rows and columns that generate imown interline transfer or frame transfer techniques. The 
at least three separate color values but only one color value image sensor 12 is covered by a color filter array (CFA) 13, 
for each photosite location, means for interpolating color joiown as the Bayer array, which is described in U.S. Pat 
values for each photosite location so that it has three y^^^ 3,971,065 and herewith incorporated by reference. In 
different color values comprising: so the Bayer geometry each color covers a photosite, or picture 

means for storing the digitized image signal; element (pixel), of the sensor. In particular, chrominance 

a processor operative with said storing means for gener- colors (red and blue) are interspersed among a checkerboard 
ating an appropriate color value missing ficom a pho- pattern of luminance colors (green). The image sens(x- 12 is 
tosite location by the interpolation of an additional exposed to image ligiht so that analog image charge infor- 
color value for such photosite locations from color 65 mation is generated in respective photosites. The charge 
values of different colors than the missing color value information is applied to an output diode 14, which converts 
at nearby photosite locations, said jH-ocessor including the charge information to analog image signals correspond- 
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ing to respective picture elements. The analog image signals An operation display panel 30 is connected to the control 

are applied to an A/D converter 16, which generates a digital processor 20 for displaying information useful In operation 

image signal ftom the analog input signal for eadi picture of the camera. Such infwmation might indude typical 

element The digital signals are appUed to an image buffer photogr^hic data, sudi as shutter speed, aperture, «posure 
18. which may a random access memoiy (RAM) with j bias, color balance (auto, tungsten, fluorescent, dayUght), 

storage capaaty for a plundity of still images. lieldfeune, low battay, low Ught. exposure modes (aperture 

Aoontrolpiocessor2Cgenerallycontrolstheinputseclion preferred, shutter preferred), and so on. Moreover, other 

^ ^ TT 7- "T^^ controlling «posure (by information unique to this type of camera is displayed. R)r 

operation by the diaphiagm and shutter (not shown) m the ^^^^^ reiliovable memory card 24 would ordinarily 

«posure section 10) by generating the honzontal and j^^ude a directory signifying the beginning and ending of 

v«bcal docks needed for dnvmg the image sensor 12 and stoned imagefnTwould show olTTdisplay panel 30 

f™^^ T r/'^'T^*^?'"°f^ f °f l^B^ ''P'''*'' «Mnaining?7 estimated to be 

for each signal segment relating to a picture dement (The remaining 

control processor 20 would ordinarily include a micropro- i< rm. • i . , 

cesser coupled with a system timing circuit.) Once a certain "^^ital signal processor 22 interpolates each still 

number of digital image signals have been accumulated in y"^"" T^^e stored in the image bi£er 18 according to the 

the image buffer 18, the stored signals are appUcd to a digital ^^terpolaUon technique shown m Fia 2. Hie mtcrpolatioo 

signal processor 22, which controls the throughput process- nussmg data yaluM at each pixel location follows the 

ing rate for the interpolation and recording section 4 of the f^^^nce shown m HG. 2; that is, first, the high frequency 

camera. The digital signal processor 22 appKes an interpo- for the missmg green" pixels (ic, the red and 

lation algorithm to the digital image signals, and sends the ^"^^ locations) are mtcrpolated to improve the lumi- 

intexpolated signals to a conventional, removable memory r^P^^^ion and, secondly, the color difference inf orma- 

card 24 via a connector 26 ^ intcipolated at the higb frequency locations by 

Since the inteipolation and related processing ordinarily „ bilinear metiiods to gena^te die other colors of the CFA 

occurs over scve^ steps, the interm^teproLts of the " f^^^^^^^^^^ 

processing algorithm aJe stored in a processing buffer 28. f ^o^^.o? techmquc is used mAe lummance section 36 

Cnieprocessingbuffer28mayalsobeconiiguredaspartof ^^.ul^''^^^^^^ of ttie system for images 

the memory sj^ce of the image buffer 18.) TTie nunS)er of ^^^.,3 • 71^"^^ T"^ 

image signals needed in the iLge buffer 18 before digital 30 Z ' ^ in^crpolatod either honzontaUy, vert^y or 

processingcanbegindependsonLtypeofprocessing^tiiat ^o-dmiensionaUy <te^^^ 

is. for a neighbAood interpolation to be^, a bX^k of ^^"^^ chrominance (red and blue) pixel locations in 

signals including at least a portion of the knage signals vertical and horizontal directions around the ''missing 

comprising a video frame must be available. Consequently, SF^^ pixeL 

in most circumstances, the interpolation may commence as 35 adaptively interpolating the '*missing 

soon as the requisite blocic of picture elements is present in gfcen" pixels is to select an interpolation method. TTie 

the buffer 18. details of tfiis process are shown in block 40 of FIG. 3. The 

The input section 2 operates at a rate commensumte with ^^^^^^ computing two composite pixel classifier 

normal operation of the camera while interpolation, which ^^"^ ^^^^ ^® horizontal direction and one 

may consume mwe time, can be relatively divorced from the 40 vertical. The term **pixel classifier** denotes a value 

inputrate-Ttieexposure section 10 exposes the image sensor confuted for the purpose of making a decision aboutfurthcr 

12 to image light for a time period dependent upon exposure P^^cessmg of the pixel information. The term "composite" 

requirements, for example, a time period between yjooo ^® dependency of the value on a multiplicity of 

second and several seconds. The image diaige is then swept c^lorplanes. In this case, the absolute value of the Laplacian 

from the photosites in the image sensor 12, converted to a 45 ^^^^ ^ absolute value of the 

digital format, and written into the image buffer 18. The gradient of the red or blue plane, depending on which was 

driving signals provided by the control processor 20 to the corresponding color in the Bayer color filter array, 

image sensor 12, the A/D converter 16 and the buffer 18 are classifier values are the compared (Block 52) and 

accordingly generated to achieve such a transfer. The pro- tested for equality. In the likely event that one value is 

cessing throughput rate of the interpoladon and recording 50 ^^"^^ than the other, the interpolation method correspond- 

scction 4 is determined by the speed of the digital signal *o *® smaller value is selected (Block 54). If the values 

processor 22. are equal, then the default interpolation method is selected 

One desirable consequence of this architecture is that the (Block 56). In cither case Block 40 is done, 

processing algorithm employed in the interpolation and The green (luma) inteipolation step (Block 44) has two 

recording section may be selected for quality treatment of 55 parts, as shown in FIG. 4. The first part (Block 80) averages 

the image rather than for throughput speed. This, of course, tbe two luminance (green) values according to the selected 

can put a delay between consecutive pictures which may interpolation method. The second part (Block 82) adds a 

affect the user, depending on the time between photogr^hic correction factor based on either red or blue neighboring 

events. This is a problem since it is well known and values depending on if the pixel in question was covered by 

understood in the field of electronic imaging that a digital 60 a red or blue filter in the Bayer color filter airay. 

still camera should provide a continuous shooting capability The red/blue (chroma) interpolation proceeds in a manner 

for a successive sequence of images. For this reason, the similar to the green (luma) interpolation described above, 

image buffer 18 shown in HG. 1 provides for storage of a The details of this process are shown in Block 46 of HG. 5. 

pluraUty of images, in effect aUowing a series of images to The process starts by computiag two conqwsite pixel clas- 
"stackup»atvideorates.The6izeofthebufferisestablished 65 sificr values (Block 60), one for the negative diagonal 

to hold enough consecutive images to cover most picture- direction and one for the positive diagonal. The term "nega- 

taking situations. tive diagonal" denotes the line of slope -1 (connecting the 
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upper left to the lower right). The tenn '^positive diagonal** 
denotes the line of slope +1 (connecting the lower left to the 
upper light). Again, these composite classifiers are found by 
adding the absolute value of the Lapladan in the green plane 
to the absolute value of the gradient in either the red or blue 
plane, depending on which color is being interpolated. 

The two classifier values are then conipared (Block 62) 
and t&sted for equality. In the likely event that one value is 
smaller than the other, the interpolation method cocrespondr- 
ing to the smaller value is selected (Block 64). If the values 
are equal, then the default interpolation method is selected 
(Block 66). In either case Block 46 is done. 

The red/blue (chroma) interpolation step (Block 48) has 
two parts, as shown in FIG. 6. In the first part (Block 70) two 
chrominance values, either red or blue depending on the 
pixel's position in the Bayer color filter array, are averaged 
according to the selected interpolation method. The second 
part (Block 72) adds a correction factor based on green 
neighboring values. 

More specifically, the following is a detailed description 
of the operation of the digital signal processor for a specific 
example using the Bayer array. 

Green Plane Interpolation 

The first pass of the interpolation fully populates the green 
color plane. The Bayer color filter array is assumed. Con- 
sider the following neighborhood. 

Al 
G2 

A3G4A5G6A7 
G8 
A9 

Gx is a green pixel and Ax is either a red pixel or a blue 
pixel. (All Ax pixels will be the same color for the entire 
neighborhood.) For simplicity, we will use the term 
"chroma" to mean either red or blue. We form the following 
classifiers. 

DH=I-A3+2A5-A7WG4-G6I 
DV=4-A1+2A5-A9W02-G8I 

These classiiiers are composed of Laplacian second-order 
terms for the diroma data and gradients for the green data. 
As such, these classifiers are sensing the high spatial fire- 
quency information present m the pixel neighborhood in the 
horizontal (DH) and vertical (DV) directions. 

We then form three predictors. 

O5H=(04+G6y2-K-A5t-2A5-A7y4 
G5V=<G2-+<38y2-K-Al+2A5-A9y4 
G5AF<G2tG4+G6K38y4+(-Al-A344A5-A7-A9y8 

These predictors are composed of aiithmBtic averages for 
the green data and q>propiiately scaled Laplacian second- 
order terms for flie chroma data. GSH is to be used when the 
preferred orientation for the interpolation is in fiie horizontal 
direction within the pixel neighborhood. Similarly, G5V is 
to be used when the preferred orientation for tiie interpola- 
tion is the vertical direction. G5A is used when there is no 
dear preference for orientation for the interpolation. 

The complete green interpolation process may now be 
expressed as bdow. 



i9,734 

6 

IFDH<DV 

TOENGS^GSH 
ELSEIFDV<DH 

THENG5-G5V 
5 ELSEG5=G5A 

The key to this process is that both the green and the 
chroma data must indicate a minimum of hig^ spatial 
tequency information for a given orientation to be chosen 
^ as file prcfored orientation for the interpolation. If there is 
a large amount of high spatial frequency information in 
either the green data or chroma data for a given orientation, 
it will inflate the value of the corresponding classifier. This, 
in turn, reduces the likelihood for that orientation to be 

15 

chosen as the preferred orientation for the interpolation. 

In practice, the green interpolation process may be 
simpMed, as below, for computational efficiency, 

IFDH<=DV 
20 TOENG5 = G5H 

ELSE05 = G3V 

This simplification is achieved by defining the horizontal 
direction as the default preferred orientation for the inter- 
25 polation when both horizontal and vertical classifiers are 
equal The number of occurrences in a typical image when 
the horizontal and vertical classifiers are equal is so small 
that this simplification generally has negligible impact on 
the image quality of the final reconstructed image. 

30 

Red and Blue (Oiroma) Interpolation 

The second pass of the interpolation fully populates the 
red and blue color planes. In U.S. Pat No. 4,642,678, issued 
Feb. 10, 1987, Cok (the disdosure of which is incorporated 
by reference herein) disclosed the chroma interpolation 
summarized below. Consider the following neighborhood 

A1G2A3 
G4C5G6 
40 A7G8A9 

Gx is a green pixel, Axis either a red or blue pixel and C5 
is tiic opposite color pixel to Ax (ic., if Ax is red tiien C5 
is blue and visa versa). Note tiiat all Qx pixels, Gl through 
G9, are assumed to be known and coincident with all 
corresponding Ax and C5 pixels. 

There are three cases. Case 1 is when the nearest neigh- 
bors to Ax are in the same column. The following predictor 
5Q is used. (A4 is used as an example.) 

A4=(Al+A7y2+<-<51+2G4-G7y2 

Case 2 is when the nearest neighbors to Ax are in the same 
row. The following predictor is used, (A2 is used as an 
ss exaiiq>le.) 

A2=<Al+A3y2+<-Gl+202-G3y2 

Case 3 is when the nearest neighbors to Ax are at the four 
coners. The following predictor is used. (A5 is used as an 
^ exanq>le.) 

A5=(Al+Aa+A7+A9y4+<-Gl-G^+4G5-G7-G9y4 

We describe an improvement for case 3. Wc form the 
following two classifiers. 

65 

DN=H31+205-09WAl-A9l 
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DP=I-G3+2G5-G7WA3-A7I 14 output diodC 

I -« J *r 1 . J ^ 16 A/D converter 

These cLassiners are composed of Laplaaan second-order jg image buffer 
terms for the green data and gradients for the chroma data. 20 control processor 

AS such, these classifiers are sensing the high spatial fre- 5 22 digital signal processor 
quency information present in the pixel neighborhood in the 24 removable memory card 
negative diagonal (DN) and positive diagonal (DP) direc- 26 connector 

u * u ^. 28 processing buffer 

We then form three predictors. ^^j^y 

10 36 luminance section 
38 chroma section 

A5Pt=<A3+A7V2+(-G^2G5-G7y2 40 select best luma interpolation 

44 interpolate missing luma values 
A5AKAl4A^A7+A9y4K-^i-G3.4G5-<57-G9y4 4^ ^^^^^ ^est chroma interpolation 

Tliesepredictorsarecomposedofarithmeticaveragesfar ^^^^^^^^^^ . , 

the chroma data and appropriately scaled Lapladan second- ^"^"^^ horizontal and vertcal composite dassifier 

order terms for the green data A5N is to be used when tiie 1 "^^fi 

preferred orientation for the interpolation is in the negative 52 classjlier test 

diagonal direction within the pixel neighborhood. Similarly, '^^f,^ mterpolation method correspondmg to the 

A5P is to be used when the preferred orientation for the , ^ . ^ . 

interpolation is the positive diagonal direction. ASA is used ^ ^^"^ ^^"^ ^J^terpolation 

when there is no clear preference for orientation for the 5!! ?. , 

interpolation. gj^t 

The coiiq)lete case 3 chroma interpolation process may ^ 
now be expressed as below. ^ 66 block 

70 average red/blue values 

IPDN<DP 7^ &ecn correction factor 

THEN A5 = A5N 80 block 

ELSE IF DP < DN 82 blOCk 

THEN A5 = ASP 30 We claim: 

1. In apparatus for utilizing a digitized image signal 
obtained from an image sensor having color photositcs 
In this process both the green and the chroma data must aligned in rows and colunms that generate at least three 
indicate a minimuin of high spatial frequency information separate color vahies but only one color value for each 
for a given orientation to be chosen as the preferred orien- 35 photosite location, said apparatus including means for inter- 
tation for the interpolation. U there is a large amount of high polating the color values for eadi photosite location so that 
spatial frequency information in either flie greoi data or each photosite has the three separate color values, said 
chroma data for a given orientation, it will inflate the value means for interpolating tiie color values comprising: 
of the corresponding classiiier. This, in turn, reduces the ^^^.^ns foi storing the digitized image signal; 

hkehhood for that orientation to be chosen as the preferred 40 • j * • 

orientation for tiie interpolation. ^ *^P*^r*= f^? ^^^f 

In practice, tiie case 3 chroma interpolation process may ^ appropnate color value missing from a pho- 

bc simplified, as below, for computational effidency. '^f mterpolation of an additional 

^ ^ ^ color value for such photosite locations from color 

IFDN<=DP values of different colors than the missing color value 

TBEN A5 = A5N ^ at nearby photosite locations, said processes: including 

ELSEA5=A5P means for obtaining Lapladan second-order values and 

gradient values in at least two image directions from 
This simplification is achieved by defining the negative nearby photositcs of the same column and row; 

diagonal direction as the default preferred orientation for the means for adding the Lapladan second-order values and 
interpolation when both negative and positive diagonal the gradient values to define a classifier and for select- 

dassifiers are equal. The number of occurrences in a typical ing a preferred orientation for the interpolation of the 

image when the negative and positive diagonal classifiers missing color value based upon the classifier, and 

are equal is so small tiiat tiiis simplification generaUy has means for interpolating tiie missing color value from 
negligible impact on the image quality of the final recon- 55 nearby nmltiple color values sdected to agree with the 
structcd image. preferred orientation. 

The invention has been described in detail with particular 2. In apparatus for processing a digitized image signal 
reference to certain preferred embodiments thereof, but it obtained from an image sensor having red, green, and blue 
will be understood that variations and modifications can be color photositcs aligned in rows and columns that generate 
effected within the spirit and scope of the invention. ^ at least three separate colw: values but only one color value 

for each photosite location, said apparatus induding means 
Parts List interpolating the color values for eadi photosite location 

2 input section so that each photosite has the three separate color values, 

4 recording section said means for interpolating comprising: 

10 exposure section 65 means for storing the digitized image signal; 

12 image sensor a processor operative with said storing means for gener- 

13 color filter array ating an a^iropriate color value missing firom a pho- 
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tosite location by the interpolatioD of an additional 8. The ^paratus of claim 1 wherein, when the prcfeiTed 

color value fox such photosite locations from color orientation is negative diagonal, a color site is green, and the 

values of different colors than the missing color value missing color v^ue is blue, adjacent negative diagonal blue 

at nearby photosite locations, said processor including ^olor values are averaged and negative diagonal green color 
means for obtaining Lapladan second-order values and 5 values including a green color site value provide a correction 

gradient values in at least two image directions from value 

nearby photosites of the same cduim and row: 9. The apparatus of data 1 wherdn, when the preferred 

means for addmg the Laplaaan second-order values and ^„-^«*„f;«„ /I « i„ «^ ^^a ♦K/«^^cc^«« 

tiic gradient vLes to define a classifier and for select- o^<^^°° vertical, a color site is red, and tiie missmg 
inglprcfcrrcd orientation for the interpolation of tiie ,o c«>l<^ vaMe is green adjacent vertical green c^^^^ 

missing color value based upon the classifier; and averaged and vertical red color values mduding a red color 

means for interpolating the missing color value from site value provide a correction vahie. 

nearby multiple color values selected to agree with the 1®* The apparatus of daim 1 wherein, when the preferred 

preferred oientation wherein neighboring color values orientation is vertical, a color site is blue, and tiie missing 

of the same color are averaged and corrected by neigh- color value is green, adjacent vertical green color values are 

boring color values of a different color value. averaged- and vertical blue color values including a blue 

3. The apparatus of daim 1 wherein, when the preferred color site value provide a correction value. 

orientation is horizontal, a color site is red, and the missing n. The ^paratus of daim 1 wherein, when the preferred 

color value is green, adjacent horizontal green color values orientation is vertical, a color site is green, and the missing 

arc averaged and horizontal red color values including a red color value is red, adjacent vertical red color values are 
color site value provide a correction vahie. ^ , ^ averaged and vertical green color values induding a green 

4. The apparatus of claim 1 whaem, when tiie preferred ^^i^^ ^^^^ ^^^^ ^ correction value. 

orientation is horizontal, a color site is blue, and the missing ^ ^ ^ ^^^^^ ^^^^ the preferred 

color value IS green, adjacent horizontal green col<^ values ^^entation is vertical, a color site is green, and the missing 

^e averaged and honzontel blue cok. values indudmg a ; vertical blue color values are 

blue color site value provide a correction value. 25 j^^-, , i - ij- 

5. The apparatus of daim 1 wherdn, when the prefarcd ^^^^ vertical green col^ values mdudmg a green 
orientation is horizontal, a color site is green, and the ^ ^^"^ P^^^i^ f correcbon value 

missing color value is red, adjacent horizontal red color apparatus of daim 1 wherein, when the prefared 

values are averaged and horizontal green color values orientationispositive diagonal, a color site is green, and the 
including a green color site value provide a correction value. 30 nussing color value is red, adjacent positive diagonal red 

6. The apparatus of claim 1 wherein, when the preferred color values are averaged and positive diagonal green color 
orientation is horizontal, a color site is green, and the values induding agreen color site value provide a correction , 
missing color value is blue, adjacent horizontal blue color vahie. 

values arc averaged and horizontd green color values 14. The apparatus of claim 1 wherein, when the preferred 

induding a green color site value provide a correction value. orientation is positive diagonal, a color site is green, and the 

7. The apparatus of daim 1 wherein, when the pr^ctred missing color value is blue, adjacent positive diagonal blue 
orientation is negative diagonal, a color site is green, and the color values are averaged and positive diagonal green color 
missing color value is red, adjacent negative diagonal red values induding a green color site value provide a correction 
color values are averaged and negative diagonal green color value. 

values induding a green color site value provide a correction 

value. ♦ ♦ * ♦ m 
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